Soccer captioning: dataset, transformer-based model, and triple-level evaluation

This work aims at generating captions for soccer videos using deep learning. The paper introduces a novel dataset, model, and triple-level evaluation. dataset consists of 22k caption-clip pairs three visual features (images, optical flow, inpainting) 500 hours SoccerNet videos. model is divided into parts: transformer learns language, ConvNets learn vision, fusion linguistic generates captions. suggested evaluation criterion captioning models covers levels: syntax (the commonly used metrics such as BLEU-score CIDEr), semantics quality descriptions domain expert), corpus diversity generated captions). shows that the has improved (from 0.07 reaching 0.18) with semantics-related losses prioritize selected words. Semantics-related utilization more (optical normalized score by 27%.